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Concerned with providing managerial assessment 
centers with oral communication criteria against which communication 
apprehension may be measured, the convergent validity of the Personal 
Report of Communication Apprehension — Organization Form (PRCA-OF) was 
tested against 23 raters' scores in five one-day developmental 
assessment centers. The PRCA-OF is a 20-item, 5-point, Likert-type, 
self-report questionnaire. The scale scores range from a minimum of 
20 to a maximum of 100. Four sets of scores were obtained: rater, 
peer, self, and supervisor. The assessment center exercises included 
a decision game, dyadic interview, and in-basket simulation. Multiple 
regression equations were used, with communication scores as the 
dependent variable, to analyze the relationship between the PRCA-OF 
scores and the other assessment center scores. Convergent validity 
was found for the PRCA-OF in both the rater and self categories. It 
was concluded that a valid measure of communication apprehension, 
specifically the PRCA-OF, should be included in managerial assessment 
centers and that training in performance evaluation should be 
mandatory. (HOD) 
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Convergent Validity and Work Applications 
of Communication Apprehension: 
An Applied Analysis in an Assessment Center Setting 

Abstract 

Convergent validity of the Personal Report of Communication 
Apprehension— Organization Form (PRCA-OF) (Scott, McCroskey, & Sheahan, 1978) 
was tested against raters' scores in five one-day developmental assessment 
centers. The subjects were 23 current first-line supervisors and 34 aspiring 
first-line supervisors. Four sets of scores were obtained: rater, peer, 
self, and supervisor. The assessment center exercises included a decision 
game, dyadic interview, and 1n-basket simulation. Convergent validity was 
found for the PRCA-OF in both the rater and self categories. 

Multiple regression equations were used, with communication apprehension 
scores as the dependent variable, to analyze the relationship between the 
PRCA-OF scores and the other assessment center scores. Both rater and self 
scores significantly contributed to the equation. The multiple correlations 
were low to moderate. 

It was concluded that a valid measure of communication apprehension, 
specifically the PRCA-OF, be included in managerial assessment centers. 
Training personnel in performance evaluation was strongly urged. It was also 
suggested that employees be involved, via self measures, in the performance 
appraisal process. The role of peer ratings was suggested as an area of 
further research, given their consistent, yet unpredictable, nature. 



ERIC 



3 



PRCA-OF Validity 
3 

Convergent Validity and Work Applications 
of Communication Apprehension; 
An Applied Analysis in an Assessment Center Setting 

Strengths and versatilities of the assessment center technique for 
personnel decisions have been demonstrated (Borman, 1982; Bray, Campbell, & 
Grant, 197s^ Bray & Grant, 1966; Hardesty & Jones, 1968; Kraut & Scott, 1972). 
As in the multitrai t-multirater matrix approach for measuring job performance 
(Campbell i Fiske, 1959; Lawler, 1967), assessment centers utilize more than 
one performance criterion (multitrait) and more than one evaluator 
(mul tirater) • Both of these techniques rely on more than one global rating 
administered by the employee's immediate supervisor for personnel decisions, 

Morton (1977) explains that assessment center design (referring to the 
dimensions and simulations) and implementation (including the selection and 
training of raters) are vital for both moral and legal obligations of the 
organization. However, much of the current research concentrates on raters* 
judgments in performance situations (Holzbach, 1978; Lyness & Cornelius, 1982; 
Sackett & Wilson, 1982) rather than the issues of dimension selection 
construction, or their validity. A large portion of the rater judgment 
research seeks to analyze phenomena related to measurement biases, such as 
strictness, leniency, or central tendency errors, and halo and horn effects. 
Some additional work has examined training effects on raters (Bernardin H 
Walter, 1977; Borman, 1979; Latham, Wexley, & Pursell, 1975), 

As for dimension choice and construction, it has been agreed that since 
assessment centers differ substantially from objective paper and pencil tests, 
the dimensions must be both organization- anu job-specific (Moses & Byham, 
1977; Norton, 1977). Thus, interorganizational measures would be expected to 
be difficult to obtain, generalizability limited, and resultant studies 
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sparse. 

This paper examines raters' final assessment center scoring of three 
exercises to test the convergent validity of communication apprehension, which 
"... is de-^ined as an individual's level of fear or anxiety associated with 
either real or anticipated communication with another person or persons " 
(McCroskey, 1977b, p. 78). A communication apprehensive individual tries to 
avoid oral communication with others, engaging in oral communication as 
infrequently as possible. For managers, tha importance of oral communication 
has been established (Klemmer i Snyder, 1972; Mintzberg, 1973; Stewart, 1967). 
In addition, if communication in assessment centers is to be validly 
measured, an instrument is needed which conjoins with other assessment center 
measures, is easy to interpret anr:, above all, valid. 

The instrument for measuring communication apprehension, the Personal 
Report of Communication Apprehension— Organization Form (PRCA-OF) (Scott, 
McCroskey, & Sheahan, 1978), has "... face validity for measuring oral 
communication apprehension in the organizational setting . . ." (p. 104). It 
also has internal reliability, concurrent validity with the PRCA (Personal 
Report of Communication Apprehension), another communication apprehension 
measure (McCroskey, 1970), and predictive validity in the organizational 
setting. 

The PRCA-OF is a twenty-item, five-point, Likert-type, self-report 
questionnaire. The scale scores range from a minimum of 20 to a maximum of 
100. The higher the score, the higher the apprehension. Raw scores of more 
than 62 call for concern. Scores higher than 72 indicate a severe problem 
(Scott et al., 1978). Conceptually, scores one or more standard deviations 
below the true mean are desirable. The lower the score, the lower the 
apprehension or fear of oral communication. It would be predicted that 
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managers score in this low range due to their high dependency on oral 
communication for initial job placement and subsequent performance success, 

T^e rationale behind the current study is that If the existing measures 
used in managerial assessment centers have an oral communication dimension 
which can be measured, then these exercise scores may serve as criteria 
against which communication apprehension may be measured. Additionally, oral 
comrtjuni cation is the medium through which exercise performance is transmitted 
and suDsequently rated. Practically, since managers so heavily rely on oral 
communication, apprehension to communicating orally would be undesirable and 
would result in low assessment center exercise scores and vice versa. This Is 
dues to the inherent, prevalent nature of oral communication. In the end, a 
valid measure of communication must be available. 

This study focuses on the scores from each rater category: rater, self, 
peer, supervisor as independent variables and the PRCA-OF score as the 
dependent variable. Convergent validity was measured and the contribution of 
each exercise scores to the communication apprehension score was examined. 
Implications for including a direct measure of communication and for training 
both raters and supervisors in organizations are made. Finally, the nature of 
this field study allows for applications of the results to the personnel 
function in work organizations. 

Me thod 

Subjects 

The subjects were 57 current {H_ = 23) and aspiring (|[ = 34) first-line 
supervisors participating in five one-day developmental assessment centers 
(Note 1) sponsored jointly by one county and one city government. The 
subjects ranged In age from 20 to 57 years (M = 34,7), Twenty-five subjects 
were male; 29 were female (missing N = 3), Race composition was almost 
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totally Caucasian (47 caucasla-.i; 1 native American Indian; missing N = 9), 

Tenure on the subnets' present jobs ranged from less than one year to a 
maximum of 16 years (M = 3.43). Education breakdown was as follows; high 
school diploma, = 5; some college, = 22; college degree, = 14; some 
advanced work, = 7; advanced degree, 3. Thus, a general demographic 
profile would be generally stated as approximately equivalent numbers of male 
and female caucasions, in the stable period of early adulthood (Levinson, 
1978), averaging 3 years on their jobs, and having some higher education. 
The organizations established a pool of 54 raters (male = 34; female 
= 20). One-day training sessions were previously provided to introduce the 
raters to the assessment center process, exercises (decision game, dyadic 
interview, in-basket exercise), the 12 dimensions (Note 2), and the rating 
scale (1 = unsatisfactory; 2 = below average; 3 = satisfactory; 4 = above 
average; 5 = outstanding). 

A total of 12 trained raters whose job ranks were above first-line 
supervisor, observed and rated subjects In each assessment center. In 
addition, each subject was rated by his/her immediate supervisor on the same 
12 dimensions as used in the assessment centers after the assessment centers 
were conducted. These ratings were completed independent of the subjects' 
assessment center performance. That Is, supervisors were neither observers 
nor raters In the assessment centers, nor were they exposed to the training 
sessions which were conducted for the assessment center raters. 
Procedure 

Each rater observed two Individuals during each exercise. Raters and 
subjects were systematically rotated to assure each rater would observe each 
subject only once. The ratio of rater to subject was 1:1. At the conclusion 
of each assessment center day the two raters negotiated on one final score for 
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each subject for each exercise. The negotiation process was procedure in the 
city's previous assessment centers and thus, for continuity for both the 
raters and the two organizations, was used during these assessment centers. 

Along with the independent supervisors* ratings and the trained raters* 
ratings of subjects, self ratings of subjects, and peer ratings for the 
decision game and the dyadic interview were obtained. All of the ratings used 
the same 12 dimensions and 5-point rating scale. 

As a final exercise in the assessment centers, subjects completed the 
self-report Personal Report of Communication Apprehension—Organization Form 
(PRCA-OF) (Scott et al., 1978). Subjects were assured their responses would 
not be released to their work organizations and were for the sole purpose of 
research. 

Results 

Communication apprehension scores ranged from 26 to 81 (N = 54). Five 
subjects (9?) scored more than 62 on the PRCA-OF. The mean communication 
apprehension score was 48 (SD = 10.3). Correlation matrices of ratings scores 
and interactive and noninteractive categories (Note 3) appear in Tables 1 and 
2. The correlation coefficients demonstrates the measures contain several 



Insert Tables 1 and 2 about here 



different aspects; i.e., accounting for the 12 dimensions (Selltiz, 
Wrightsman, & Cook, 1976). All but two of the significant findings (peer 
interactive and rater noninteractive shown in Table 2) include the self 
category as one of the correlation coefficient variables. 

Table 1 shows one significant correlation between self and peer final 
scores. Table 2 shows six significant correlations of the following pairs of 
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scores: peer interactive and rater interactive; self interactive and rater 
Interactive; self interactive and peer interactive; rater noninteracti ve and 
rater interactive, and self noninteracti ve and both interactive scores of peer 
and self. 

In order to determine the predictive utility of the exercise scores (Note 
4) on communication, communication apprehension scores were regressed on the 
set of predictor variables as shown in Table 3. Rater scores and self scores 



Insert Table 3 about here 



each made a significant contribution {£ < .01 and £ < ,005, respectively) in 

the predicted direction {Note 5) to explained variance In communication 

apprehension scores. Peer scores also contributed significantly (£ < .05) but 

not in the predicted Inverse direction. Supervisor scores did not enter the 

equation at a significant level. The multiple correlation (R = ,55) was 

highly significant {£ < ,005). The proportion of variance explained by the 

set of predictor variables was moderate { R^^ = .30). 

Communication apprehension scores correlated with rater scores (r = -.41, 

£ < .001) and self scores {r = -,36, £ < .005) at a moderate level in the 

predicted direction, demonstrating convergent validity. Although 

communication apprehension scores correlated with peer scores significantly at 

a moderate level (£ = .44, £ < .001) it was not in the predicted direction; 

the relationship was positive, and validity of the measure can not be 

determined. The correlation between communication apprehension scores and 
supervisor scores was not significant. These findings are consistent with the 

findings of the regression equation. 

The exercises were combined to create the two categories of Interactive 
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and noninteractive exercises. With the PRCA-OF being a measure of oral 
communication, it was hypothesized that PRCA-OF scores could be predicted from 
the independent variables of rating scores as dichotomized into these two 
categories. This also allows for a summary analysis more easily interpretable 
by personnel managers or administrators, a general analysis of the data 
itself, and a measure of the convergent validity of communication apprehension 
scores. 

Table 4 presents the regression analysis of communication apprehension 
scores regressed on interactive and noninteractive scores as rated by raters, 
peers, self, and independent supervisor scores—the last of which were not 



Insert Table 4 about here 



dichotomized into interactive and noninteractive scores. Peer interactive 

scores were significant (£ < .OOS) but, as in the above analyses, not in the 

predicted inverse direction. None of the other scores entered into the 

equation at a significant level. The multiple correlation (R^ = ,51) was 

significant (£ < ,01), The proportion of variance explained by the set of 

independent variables (R^ = ,26) was at a moderately low level. 

Communication apprehension scores correlated with both rater interactive 

scores = -,28) and self noninteractive scores (r_ = -,27) at identical 

significant levels (£ < .05), As in the above regression analysis, convergent 

validity may be concluded for the rater and self categories. The correlation 

between communication apprehension scores and both self interactive scores and 

rater noninteractive scores was not significant. Peer interactive scores and 
supervisor scores did not correlate with communication apprehension scores at 

a significant level {nor in the predicted direction). 
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Discussion and Implications 
This study tested the convergent validity of the Personal Report of 
Communication Apprehension— Organization Form (PRCA-OF) (Scott et al • , 1978) 
in light of its use in managerial assessment centers* Several findings 
emerged* First, communication apprehension was found to be significantly 
inversely correlated with rater and self scores, showing convergent validity 
of the PRCA-OF* Similar findings were evidert for' the interactive and 
noninteractive types of tests; rater interactive scores and self 
non interactive scores significantly negatively correlated with communication 
apprehension scores. 

Peer scores (excluding peer ir-teractive scores) significantly contributed 
to and correlated with communication appreht-usion scores (with the exception 
of peer interactive scores, in the correlational analysis) but not in the 
dirjsction which was hypothesized* This finding is contrary to McCroskey and 
Richmond (1976), but conceptually consistent with Kraut (1975) who found peer 
ratings to be a mobility predictor while rater ratings were found to be a 
concurrent measure of employee performance* Roadman (1964) concurs with the 
mobility concept, explaining the importance of peer prediction to industry* 
In the present study, peer scores consistently defied the predicted inverse 
relationship* This finding supports the use of peer scores for uses other 
than those in this study, and the convergent validity measure of the PRCA-OF 
must be further tested* 

As for prediction, self scores emerged as the strongest predictor of 
communiCc>tion apprehension scores, and rater scores as second in their 
contribution* Peer scores significantly contributed to the equation but 
again, not in the predicted direction* In the interactive and noninteractive 
categories, none of the independent variables predicted communication 
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apprehension scores as hypothesized. However, the combined effect of the 
independent variables (each multiple correlation) in both regression equations 
were moderate, with their respective variances explaining a reasonable 
proportion of the dependent measure of communication apprehension. 

The final rating scores were statistically more sign^^-^icant (R^ = .55; 
£ < .005) than the interactive and noninteractive scores (R^ = .55; £ < .01) 
(Winch & Campbell, 1969) but the contribution of each set of independent 
variables was identical. The utility of the combined effect of each set of 
independent variables shows the importance of multi raters and multi traits as 
they are combined for use in assessment centers. It also demonstrates the 
predictive nature of some of the final rating scores but not these scores as 
combined into interactive and noninteractive categories. 

Thp number of high communication apprehensives in this sample is roughly 
equivalent to the number found in a similar assessment center setting 
(Kandelman, 1980) and approximations of other settings (McCroskey, 1977a). 
From these results, the PRCA-OF may be suggested, with caution as this point 
of the research, for use as an additional interactive measure in assessment 
centers. F^r the time being, analyses should be restricted to the rater and 
self categories. 

The importance of measuring communication is obvious. This research has 
shown that communication apprehension can be predicted from final rating 
scores. The measure of communication apprehension (PRCA-OF) is a valid one. 
However, the measure of communication apprehension as predicted from 
interactive and noninteractive rater and self scores is virtually nonexistent. 
The results suggest the validity of the measure of communication apprehension 
as measured against the judgments of the rater and self scores. 

The major implications of this study are clear. First, a valid measure 
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of communication should be included when evaluating managerial level 
employees. The significant bivariate correlations of this study demonstrate 
the convergent validity of the measure of the PRCA-OF for measuring oral 
communication in assessment centers. This measure has been suggested, 
however, with caution. Other similar measures are available (Daly, 1978) and 
may be used in both the assessment center setting and in conjunction with 
traditional performance evaluations. 

Second, training in performance evaluation Is mandatory. Research on 
evaluator trainir*. is shown to minimize rating errors (Bernardin & Walter, 
1977; Borman, 1979; Hclzbach, 1978; Latham etal., 1975). The training 
offered the raters of this study focused on observation of behaviors and 
exercise scoring on the 12 dimensions. No training of interactive or 
noninteractive category distinction was given. These latter categories 
yielded no predictive power, barring the peer score which predicted in the 
wrong direction. As for the independent supervisor ratings, they neither 
explained nor correlated with the communication apprehension measure, where 
the trained rater ratings do just that. In this study, supervisors did not 
receive the standardized training giv^en to the assessment center raters. 
Neither of the organizations otherwise offered performance training to it^ 
supervisors. 

A third implication is to Involve the employee, via self measures, in 
his/her performance appraisal {Holzbach, 1978). It is a difficult step for 
many managers (Patz, 1975). However, as evidenced from this study, including 
employees in the appraisal jf their job performance adds valuable information 
to it. Employees hold the most information about their abilities as well as 
lack of them; particularly the skill of oral communication. 

Finally, the role of peer ratings in performance evaluations must be 
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further examined* Although the demographic profile of the subjects used in 
this study is narrow and the work organizations is that of government, the 
results of this work taken with others as previously cited, point to the 
peculiar role of peer ratings. As Mumford (1983) states, . • the validity 
of peer evaluations represents something of an enigma" (p* 867) • It is 
necessary to discover whether peer ratings are solely reliable as mobility 
measures or if additional uses may be made of them. This research is called 
for. 

In addition, the categorization of interactive and noninteractive and 
their measurement should be developed further especially when measuring both 
oral and written communication skills. Although this dichotomy, is simple both 
conceptually and practically for measurement in work organizations, it may 
result in categories too broad to yield the necessary sensitive measurements, 
Nevertheless, where oral communication is vital to the supervisory function, 
written communication is no less important. First, an awareness of and 
subsequently a measure of, the interactive and noninteractive categories may 
be helpful in determining what behaviors to observe and which skills to test 
for any given managerial level job. 
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Footnotes 

^The purpose of a developmental assessment center is to evaluate 
candidates' job-related strengths and weaknesses for training and/or 
development. In this study, current first-line supervisor held the rank of 
first line supervisors in the organization. Aspiring first-line supervisors 
were those whose next promotion would be to the rank of first-line supervisor. 

The twelve dimensions were: communication skills, judgment, 
decisiveness and assertiveness, problem solving and decision making, 
impartiality, recognition of others* contributions, interest and sensitivity 
to others, interest in work, concern with others' work, creativity, planning 
and organizing of own work, planning and organizing work for others. 

^Interactive exercises is defined as those exercises which required 
subjects* engaging in oral communication as a means to complete the exercises. 
The interactive exercises were the decision game and the dyadic interview* 
The noninteractive exercise category, conversely, did not require subjects to 
orally communicate. It included the in-basket exercise. These two categories 
were created to align with the organizations* rating categories of oral 
communication skills and written communication skills. 

^The exercises included decision game, dyadic interview, and in-basket 
simulation. 

^Since the scoring goal is an increase in exercise scores and a decrease 
in communication apprehension scores (and vice versa), an inverse relationship 
is predicted. 
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Table 1 

Correlation Matrix of Final Rating Scores 



(N = 57) 



Rater 

Peer 

Self 



Peer 
.17 



Self 

.18 

.44* 



Supervisor 
-.00 
.02 
-.04 



*£ < .001, two-tailed 
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Table 2 

Correlation Matrix of Interactive, Noninteractive, and Supervisor Scores 



N = 57 



Peer 



Self 



Rater 



Self 



Interactive Interactive Noninteractive Noninteractive Supervisor 



Rater 

Interactive .45** 
Peer 

Interactive 
Self 

Interactive 
Rater 

Noninteractive 
Self 

Noninteractive 



.52** 
.63** 



.27* 
-.04 

.03 



.19 
.29* 

.48** 

.23 



-.02 
-.03 

-.03 

-.09 

.11 



Note: All significance levels are two-tailed 
*p < .05 
**p < .001 
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Table 3 

Correlation and Regression of Communication 
Apprehension Scores and Final Rating Scores (N^ = 54) 





Communication Apprehension 






Partial 






IndGOGndent Variable 

All VI ^ 1^ ^ 1 1 vl ^ 1 1 V f vl 1 V ^Jk 1 ^ 


Beta 


F-Ratio 




Rater Scores 


-.344 


7.69** 


-•41**** 


Peer Scores 


.301 


4.88* 




Self Scores 


-.418 


8.83*** 


-.36*** 


Supervisor Scores 


.089 


.55 


.04 




R = .55; df = 


4.49; 






F = 5.39; £ < 


.005 





Note. All significance levels are one-tailed. 



*£ < .05 
**£ < .01 
***£ < .005 
****£ < .001 
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Table 4 

Correlation and Regression of Communication Apprehension 
Scores and Interactive and Noninteractive Scores {N^ = 54) 

Communication Apprehension 



Partial 

Independent Variable Beta F-Ratto 



Rater Interactive Scores - .289 3.07 - .28* 

Peer Interactive Scores .50 8.40** .11 

Self Interactive Scores - .249 1.61 - .20 

Rater h'onis ^^^ractive Scores .000 0.00 - .18 

Self Noninteractive Scores - .252 2.92 • .27* 

Supervisor Scores .09 .51 .03 

R = .51; df = 6,47; 

F = 2.73; £ < .01 

Note . All significance levels are one- tailed. 
*£ < .05 
**p < .005 
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